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ABSTRACT 



The Enterprise Florida Jobs and Education Partnership was 
developed to create and maintain a highly skilled work force able to respond 
to rapidly changing technology and market opportunities essential to a 
competitive Florida economy. A review of the partnership found the following: 
(1) the partnership has adopted a research design as required by statute, 
although outcome measures were developed for only three of its goals; (2) the 
partnership is coordinating with the Florida Education and Training Placement 
Information Program to obtain data for evaluating the program's outcomes; (3) 
the partnership has carried out various activities in coordinating and 
overseeing state work force training programs, such as development and 
oversight for the Performance-based Incentive Funding Program and 
coordinating efforts to identify occupations that are in demand in Florida's 
economy and related job training programs designed to prepare individuals for 
these occupations; and (4) it is still too early to determine if these 
activities will translate into improved outcomes for the disadvantaged. The 
following recommendations were made: develop outcome measures to evaluate the 
achievement of all its statutory goals; ensure the availability of all data 
needed to evaluate job training programs,* coordinate with other agencies that 
administer work force development programs; and develop a system for 
monitoring performance of community colleges and school districts to ensure 
that partnership funds are used for maximum benefit. (KC) 
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Purpose of Review 



The Office of Program Policy Analysis and 
Government Accountability (OPPAGA) conducted 
this review of the Enterprise Florida Jobs and 
Education Partnership (the Partnership) pursuant to 
s. 288.0475(10), F.S. 1 The Partnership is statutorily 
required to develop a research design that provides 
for a quantitative evaluation of its performance. The 
objectives of our review were to evaluate the 
Partnership using its research design and report on its 
implementation and progress in achieving statutory 
goals. 



1 The Office of the Program Policy Analysis and Government 
Accountability is a unit of the Office of the Auditor General, but 
operates independently and reports to the Legislature. 



: 1 Background 



The Legislature established the Enterprise Florida Jobs 
and Education Partnership in 1994 as an affiliate of 
Enterprise Florida, Inc. Enterprise Florida, Inc. 
focuses on creating higher-paying jobs through 
technology innovation and capital formation, (See 
Reports No. 95-07, 95-08, and 95-09.) The purpose 
of the Partnership is to create and maintain a highly- 
skilled workforce able to respond to rapidly changing 
technology and market opportunities essential to a 
competitive Florida economy. The Legislature 
established the following goals for the Partnership: 

■ Provide for a skilled workforce to enable Florida 
to compete in a global economy; 

■ Respond to changes in technology and to 
emerging industries; 

■ Promote the development of market-driven 
programs identified in the Occupational 
Forecasting Conference (s. 216.136, F.S.); 




■ Base evaluation of success on student and 
participant outcomes rather than processes; 

■ Coordinate state, federal, local, and private 
funds for maximum impact; and 

■ Encourage the training of disadvantaged groups 
such as those with disabilities, economic 
disadvantages, dislocated workers, and public 
assistance recipients. 

The Partnership is to accomplish these goals by 
coordinating and overseeing approximately 48 state 
workforce training programs. The Partnership does 
not directly administer state workforce training 
programs; the vast majority are administered by: the 
Department of Education through postsecondary 
vocational training programs offered by school 
districts and community colleges; the Department of 
Labor and Employment Security's training programs 
for individuals who have difficulty obtaining gainful 
employment due to various social, economic, or 
physical disabilities; and the Department of 
Commerce's Quick Response Training Program 
designed to meet short-term training needs of new 
and expanding companies. 

The Partnership is also responsible for developing 
and overseeing the implementation of a program to 
provide financial incentives to postsecondary 
vocational training programs that are successful in 
improving student outcomes. Enterprise Florida's 
Strategic Plan for 1993-94 indicates that the key to 
the Partnership's success will be the establishment of 
performance-based training that will support the high- 
skill, high-wage industries necessary to achieve 
Enterprise Florida's mission. 

The Partnership is governed by a Board of Directors. 
The Board appointed a president effective September 
15, 1995. In 1995-96, the Legislature appropriated 
$226,892 for the administration of the Partnership. 
Funds were expended for two full-time employees 
and to support Board activities. 2 The Legislature 
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also allocated. $30 miljion of existing job training 
funds to provide incentives for community colleges 
and school districts to improve student outcomes. 



2 In 1994-95, the Legislature did not appropriate funds for the 
administration of the Jobs and Education Partnership. However, 
Enterprise Florida transferred $208,307 from the Innovation 
Partnership to cover its administrative costs. 



Research Design 



The Partnership Has Adopted a Research Design as 
Required by Statute. The Partnership also 
Developed Performance Measures for Its Six 
Statutory Goals. However, Outcome Measures 
Were Developed for Only Three of the Goals. In 
Addition, Baselines and Expected Future 
Performance Levels Were Not Identified for All 
Outcome Measures. 

The Partnership adopted its research design in August 
1994 after consulting with OPPAGA. In its research 
design, the Partnership developed performance 
measures for each of its six statutory goals. 
However, the Partnership specified outcome 
measures for evaluating the achievement of only 
three of its statutory goals, (i.e., promoting market- 
driven workforce training, evaluating the success of 
training programs on the basis of student outcomes, 
and encouraging the participation of target 
populations). (See Exhibit 1.) Examples of outcome 
measures developed by the Partnership are 
placement rates of completers of training programs 
and Department of Labor and Employment Security 
job training programs; wages of training completers 
one and two years after placement in instruction- 
related jobs; and the proportion of program 
completers who receive public assistance. The 
Partnership needs to develop outcome measures for 
all of its goals to evaluate its performance. 



Exhibit 1: Performance Measures for Partnership Goals 



Performance Measures 


Partnership Goals 


Inputs 


Outputs Outcomes 


• Providing for a Skilled Workforce 


✓ 


✓ 


• Responding to Technology Changes 




✓ 


• Encouraging Development of 
Market-Driven Programs 


✓ 




• Basing Success on Outcomes 


✓ 


✓ ✓ 


• Coordinating Training Funds 

• Encouraging Target Group Participation 


✓ 


✓ ✓ 



Source: Jobs and Education Partnership Research Design. 
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The Partnership has also developed baseline 
measures, expected levels of future performance, and 
timeframes for accomplishment for some of the 
outcome measures. (See - Exhibit 2.) Baseline 
measures are necessary in order for a program to 
evaluate changes in performance that occur from one 
year to the next. Expected performance levels and 
timeframes provide decision-makers with criteria to 
judge program success. 



Exhibit 2: Baselines, Future Performance Levels, and 

Timeframes for Outcome Measures 



Outcome Measures 


Baselines 


Future 

Performance 

Levels 


Timeframes 


• Placement Rates 


✓ 


✓ 


✓ 


• Wages at Placement 


✓ 


✓ . 


✓ 


• Wages l & 2 Years 
After Placement 


✓ 






• Completers On Public 
Assistance 


• Productivity (All 


✓ 


✓ 


✓ 


Student Outcomes) 


• Employer Satisfaction 


✓ 







Source: Jobs and Education Partnership Research Design. 



The Partnership's baseline measures for placement 
rates and wages are based on the Department of 
Education's Florida Education and Training 
Placement Information Program (FETPIP) data on 
individuals who completed postsecondary vocational 
and job training programs in 1992-93. Expected 
future performance levels and timeframes include a 
3% per year increase in placement rates beginning in 
fiscal year 1995-96. Further, after three years of 
operation, the Partnership expects a 15% increase in 
wages at placement and a 30% increase in 
productivity, which is a composite of all student 
outcomes. The Partnership is planning to use wage 
data collected annually by FETPIP for the period of 
October through December. 



Implementation an d results 



Data Is Not Yet Available to Evaluate the 
Partnership's Performance Using Its Outcome 
Measures. However, the Partnership Is 
Coordinating With the Florida Education and 
Training Placement Information Program (FETPIP) to 
Obtain Data for Evaluating These Outcomes. 
FETPIP Has Also Collected Data That Assisted the 
Partnership in Establishing Baselines and Timeframes 
for Expected Future Performance. 
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Because data is not yet available, we cannot evaluate 
the outcomes reported in the Partnership's research 
design. The Partnership is coordinating with FETPIP 
to obtain data for evaluating these outcomes. FETPIP 
compiles data annually on individuals who complete 
vocational and job training programs including 
placement rates, percent receiving welfare, and 
wages earned during the quarter ending in 
December. Outcome data for 1994-95 program 

participants will not be available until FETPIP 
compiles wage and public assistance data for 
October through December 1995. 

The Partnership can also use the FETPIP data to 
establish additional baseline measures. For example, 
the Partnership has not established baseline measures 
or expected performance levels for its goal of 
reducing the level of dependence on public 
assistance. FETPIP follow-up data on 1993-94 

completers of vocational education programs and 
those placed by employment assistance programs 
showed the percentage who received public 
assistance in the fourth quarter of 1994 varied by 
program, ranging from 4% for community colleges 
degree programs to 74% for Project Independence. 
This baseline data will assist the Partnership in 
evaluating the extent to which participation in 
employment training programs reduces dependence 
on public assistance. 

The Partnership Has Carried Out Various Activities 
in Coordinating and Overseeing State Workforce 
Training Programs. It Developed and Provided 
Oversight for the Performance-Based Incentive 
Funding (PBIF) Program. It Also Coordinated Efforts 
to Identify Occupations That Are in Demand in 
Florida's Economy and Related Job Training 
Programs Designed to Prepare Individuals for These 
Occupations. 

Since its creation, the Partnership has carried out 
various activities in achieving its goal of coordinating 
and overseeing state workforce training programs. 
The Partnership developed and coordinated the 
implementation of the Performance-Based Incentive 
Funding Program (PBIF) which is designed to 
improve public vocational training programs. As a 
result of the Partnership's efforts, 20 of 28 
community colleges and 19 school districts 
participated in the PBIF Program in fiscal years 1994- 
95 and 1995-96. 3 

Preliminary data reported by community colleges and 
school districts indicate that the Partnership's PBIF 
Program may be producing some positive short-term 
results. Based on information reported through 
October 1995, enrollments of disadvantaged students 
in training programs who qualify for performance- 
based incentives increased 47%, from 29,235 in 



1993-94 tp 42,869 as of October 1995. Increased 
enrollments may be partially attributed to improved 
reporting by community colleges and school districts 
since fiscal year 1994-95 was the first year the PBIF 
Program provided incentive awards for enrollments 
of disadvantaged students. 

The Partnership also coordinated with the 
Occupational Forecasting Conference, Department of 
Education, and Department of Commerce to identify 
occupations that are in demand in Florida's economy 
and vocational training programs designed to prepare 
people for these occupations. The criteria for a 
market-demand occupation is a minimum of fifty job 
openings statewide and with opportunities for full- 
time employment and a minimum entry wage of 
$7.00 per hour in 1994-95 increased to $7.50 in 
1995-96. After the . Occupational Forecasting 
Conference identifies market-demand occupations, 
the Partnership approves the occupations that qualify 
for the PBIF Program. In 1994-95, the Partnership 
approved 122 occupations for performance 
incentives and identified 173 public vocational 
training programs that prepare students for these 
occupations. In 1995-96, the Partnership approved 
152 occupations and identified 242 vocational 
training programs that prepared students for these 
occupations. The PBIF Program awarded community 
colleges and school districts approximately $3.5 
million for performance incentives for training 
students for market-demand occupations in 1994-95. 



3 Of the 67 school districts in the state, 29 have one or more 
vocational training centers; 17 of these elected to participate in the 
PBIF Program. In addition, 2 school districts that do not have 
vocational training centers are participating in the Program based on 
outcomes from employment training in their adult education 
programs. 



Observations 



■ Although PBIF Program outcomes show improved 
performance in serving disadvantaged groups, it is 
still too early to make conclusive judgments as to 
whether this progress will translate into improved 
outcomes for the disadvantaged. Since some 
training programs take as much as two years to 
complete, it will be several more years before 
outcomes (such as becoming more financially self- 
sufficient or less reliant on public assistance) can 
be validly assessed. 

■ Although the Partnership's research design did not 
include strategies for identifying and eliminating 
poor performing programs, the Partnership has 
begun to address these issues in its redesign of the 
state's workforce delivery system. The Partnership 



plans to evaluate employment outcomes annually 
by service delivery area and report on the overall 
performance of each area. In its November 1995 
meeting, the Partnership established 22 workforce 
delivery areas for the state and will require each 
to apply for a charter to provide workforce 
development programs. Beginning at the end of 
fiscal year 1995-96, the Partnership will identify 
poor performing service delivery areas and will 
require workforce development boards to develop 
and implement diagnostic procedures for 
improving outcomes. If performance does not 
increase within a specified timeframe, the 
Partnership will revoke the charter. However, the 
Partnership has not yet developed consistent 
measures that are needed to evaluate the state's 
workforce development programs. The various 
employment training programs currently have 
different definitions of performance measures such 
as program completion and job placement. 
Consistent performance measures are essential for 
valid outcomes. 

■ In Report No. 94-41, issued March 1995, we 
noted that since the Partnership's PBIF Program is 
designed to pay incentives only when outcomes 
are confirmed, it is possible that earning incentive 
dollars could take two to three years. We also 
cautioned that this could place JTPA funds 
allocated, but not expended for incentives, at risk 
of reverting back to the federal government. This 
scenario occurred in 1994-95 when the PBIF 
Program expended only $1 million of the $17 
million in JTPA Title III funds for dislocated 
workers. Unexpended Title III funds exceed the 
federal limit that the state can carry funds forward 
to the next fiscal year. Consequently, next year's 
allocations will be reduced by $3.8 million. 



RECOMMENpATIONS 



We recommend that the Partnership: 

■ Develop outcome measures to evaluate the 
achievement of all of its statutory goals and 
baseline measures, expected levels of future 
performance, and timeframes for accomplishment 
so it can provide the Legislature with the 
information necessary for making informed 
decisions regarding expenditure of state training 
funds; 



■ Ensure the availability of all data needed to 
evaluate job training programs; 

■ Coordinate with FETPIP and agencies that 
administer workforce development programs to 
establish a system for evaluating job training 
programs and criteria for acceptable levels of 
performance. The system should provide a 
mechanism for evaluating all major employment 
training programs and for eliminating poor 
performing programs. The Partnership needs to 
develop consistent performance measures for valid 
comparisons of outcomes across programs; and 

■ Develop a system for monitoring performance of 
community colleges and school districts to ensure 
that PBIF Program funds are used for maximum 
benefit. 



Enterprise Florida Jobs and Education 
Partnership Response 



The President of Enterprise Florida Jobs and 
Education Partnership, in his written response to our 
review, indicated that: 

"We are in overall agreement with the findings and 
recommendations of the report, and will work with 
your office in refining the baselines and performance 
measures in our research design. 

"We are very proud of the accomplishments of the 
Jobs and Education Partnership during its first year of 
operations. Refining the local validation and review 
of the occupational forecasting data, and utilizing 
that as a foundation for the Performance Based 
Incentive Funding program, occupied much of the 
fiscal year. However, the Partnership was also 
engaged in other important activities. These 
included a review of the organizational structure of 
the Florida Departments of Commerce and Labor 
Employment Security, and a CEO consultation. The 
Commerce and Labor review contributed to our 
future work in the redesign of Florida's Workforce 
Development System. The CEO consultation, co- 
hosted by Commissioner Frank Brogan and Chairman 
Schultz, brought together the chief executive officers 
from the largest corporations in Florida. The CEOs 
identified those attributes that make entry level 
employees successful in their businesses. The 
consultation resulted in strategies to incorporate these 
essential competencies into our public school 
curriculum." 



This review was conducted in accordance with generally accepted government auditing standards and included appropriate performance 
auditing and evaluation methods. Copies of this report in alternate accessible format may be obtained by contacting Report Production at 
(904) 488-0021 or TAX (904) 487-3804. 

Review Supervised by: jane Fletcher 





Review Conducted by: Dorothy Cray 
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